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BACKGROUND 

In October of 2005, EdSource and its partners from Stanford University, U.C. Berkeley, and American 
Institutes for Research issued the Initial Report of Findings for its study: Similar Students, Different 
Results: Why Do Some Schools Do Better? (Go to http://www.edsource.org for the full Initial Report of 
Findings and to see the surveys.) 

This research project surveyed 80% of the K-5 classroom teachers (over 5,500) and every principal at 
257 elementary schools in California serving large percentages of low-income, minority, and English 
Learner students. Specifically, 98% percent of the schools in our sample qualified for Title I funding. 

Statewide, California's elementary school population in 2004-05 included 32% English Learners, 50% 
Hispanics, 8% African Americans, 29% whites, 8% Asians, and 57% low-income students (based on 
participation in the free and reduced price meal program). The schools in our sample fell between the 25 th 
to 35 th percentile band of the state’s Schools Characteristics Index, and thus, in general, had higher 
proportions of EL, Hispanic, and low-income students. The total student population served by the schools 
participating in our sample included 42% English Learners, 66% Hispanics, 8% African Americans, 15% 
whites, 6% Asian, and 78% low-income students. These percentages varied from school to school. 

The goal was to identify which specific school practices are most strongly associated with higher levels of 
student achievement, as measured by the school’s Academic Performance Index (API). In California, 
elementary school API scores and rankings are based upon student test scores on the year-end California 
Standards Tests (CSTs), which measure how well students in the school are mastering California’s 2nd to 
5th grade academic content standards primarily in English language arts (ELA) and math. 

Our surveys of principals and teachers asked over 300 items each, covering seven broad domains of 
effective schooling practices. The domains were drawn from previous research in this field, but were 
updated with specific questions reflecting California’s current standards-based policies and reforms. The 
survey instruments were reviewed externally by educators and researchers, then field-tested in eleven 
elementary schools before being administered as a full-scale survey. 





Extensive analysis of the survey findings used regression analysis to determine which school and district 
practices, as reported by teachers and principals, are more closely related (i.e., highly correlated) to higher 
school API scores. The practices found to be most strongly associated with high performance fell under 
the following four broad domains: 

• Prioritizing Improving Student 
Achievement 

• Implementing a Coherent, 

Standards-Based Curriculum and 
Instructional Program 

• Using Assessment Data to Improve 
Student Achievement and 
Instruction 

• Ensuring Availability of 
Instructional Resources 

FURTHER INVESTIGATION OF 
STANDARDS-BASED 
CURRICULUM 

One of the strongest correlations identified 
in our Initial Finding was the positive 
relationship between a school’s API score 
and having a coherent curriculum and 
instructional program based upon 
California’s grade-by-grade academic 
standards. This particular curriculum and 
instruction domain was measured using 48 
items on the teacher and the principal 
surveys. 1 

In the months since the study’s Initial 
Findings were released, the research team 
has taken a closer look at the components 
that make up this very important effective 
school practice domain. 

The “Core Instruction” section of the 
Similar Students teacher survey included 
three different questions about the school’s 
math and English language arts curriculum 
programs: the amount of instructional time 
spent daily on each subject; the frequency 
with which each of a list of different 
curriculum programs was used by the 
teacher; and the types and frequency of student assessments given by the teacher. In addition, this section 
asked the teacher to voice agreement or disagreement with 15 statements related to the school’s 
instructional programs in math and English language arts. 



STATE POLICY ROLE IN CURRICULUM CHOICE 
IN CALIFORNIA 

According to California’s constitution, “the State Board of 
Education shall adopt textbooks for use in grades one through 
eight throughout the state.” 

The State Board (SBE) currently approves curriculum program 
materials on a staggered schedule, every six years for 
reading/language arts/English language development; math; 
science; and history/social studies. 

For each subject area, the SBE determines and announces the 
specific criteria it will use to evaluate curriculum program 
materials submitted for consideration by the different publishers. 
One criterion in recent years has been the extent and manner in 
which the proposed curriculum texts and supporting materials 
align with California’s grade-by-grade academic content 
standards. Curriculum program materials from different 
publishers may also differ in the extent to which specific content 
areas are covered, the number and type of classroom diagnostic 
tests they provide, the manner in which they address the needs 
of English Learners, the amount and kind of direction and 
training they provide for teachers, and more. 

After determining which curriculum packages it will adopt for a 
specific subject area, California’s SBE provides school districts 
with a menu of approved or state adopted texts and 
instructional materials in that subject for each grade level. 

Guided by this list, local school boards and district 
superintendents then use their own criteria to determine which 
of the approved materials might best meet their needs. The 
state expects districts to choose from the list of state-adopted 
materials and provides categorical funds for that purpose. 
Districts can request a waiver from the state if they can make a 
case that non-adopted materials might better serve the academic 
needs of their students. District choices around subject area 
curriculum programs are important and complex, and generally 
take into consideration many local factors. 



1 A detailed account of the items from the principal and teacher surveys that were combined to create composite 
variables under the domain Implementing a Coherent, Standards-Based Curriculum and Instructional Program is 
available upon request. 
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FINDINGS: ELA CURRICULUM CHOICE AND INTENSITY OF USE BY SCHOOLS 

In 2002, California’s State Board of Education (SBE) adopted only two English language arts curriculum 
packages, one published by Open Court and one by Eloughton Mifflin. A preliminary ELA adoption a 
few years before had included some additional choices, which we believed might still be in use in some 
schools. 

In our survey, K-5 classroom teachers were asked: “How frequently do you use the following language 
arts curriculum programs in your classroom?” They could choose all that apply from the following list (in 
alphabetical order): Harcourt Brace Spelling, Houghton Mifflin, McGraw Hill/Macmillan’s Reading, 
Open Court, Phonics SRA, Scholastic Phonics, Scott Foresman, Write Source Language Program, and 
Other. 

Teachers were asked to indicate how often, on the following range of frequencies, they used each 
program: daily, once or twice a week, once or twice a month, a few times a year, or never. 2 Responses to 
each of these items were averaged across all K-5 classroom teacher respondents within a school to obtain 
indicators of curriculum usage intensity at that school. 

When the data were analyzed, strikingly clear patterns emerged. With regard to use of English language 
arts curriculum programs, the schools in the study sample tended to fall into two main groups: they were 
either high intensity users of Open Court or high intensity users of Houghton Mifflin, 3 two main English 
language arts curriculum packages currently approved by the State Board of Education. Although a 
number of schools used some additional supplemental programs, Figure 1 shows that 94% of the schools 
could be classified unambiguously as relying predominantly on Open Court (27.2%) or on Houghton 
Mifflin (66.8%). 



Figure 1 - Cross Tabulation of Schools by High Intensity Language Arts Curriculum Usage 




Houghton Mifflin High 
Intensity Usage = No 


Houghton Mifflin High 
Intensity Usage = Yes 


Total 


Open Court High Intensity 


16 


177 


193 


Usage = No 


6.0% 


66.8% 


72.8% 


Open Court High Intensity 


72 


0 


72 


Usage = Yes 


27.2% 


0% 


27.2% 


Total 


88 

33.2% 


177 

66.8% 


265 

100.0% 



With regard to Open Court, 27.2% of the total school sample in the study (72 schools) could be classified 
as high-intensity users of this curriculum. In fact, in 21.5% of the total school sample every teacher in the 
school reported using it daily. With regard to Houghton Mifflin, 66.8% of the total sample of schools 
could be classified as high intensity users of this program; however, in only 34.3% of the total school 
sample did eveiy teacher report daily use of the Houghton Mifflin language arts program. 

Frequency of use within schools varied greatly between the two categories of high-intensity users. In 
79.2% of the high-intensity Open Court schools, all teachers reported using the program daily; whereas, 
in only 51.4% of the high-intensity Houghton Mifflin schools did all teachers report daily use. In other 



2 The survey items related to the frequency of language arts and math curricula usages are reproduced in 
Appendix 1. 

3 On a one-to-five scale, with 1 = Never and 5 = Daily, high-intensity Open Court usage was indicated by an 
average teacher response of 4.5 or higher. High-intensity Houghton Mifflin usage was indicated by an average 
teacher response of 3.9 or higher. These thresholds were detennined by inspecting the distributions over schools of 
the average teacher responses. Appendices 2A and 2B contain the descriptive statistics of the within-school average 
language arts and math curriculum usage frequency broken out by the high intensity usage categories specified in 
Figures 1 and 2. Complete distributions of the school-average teacher usage intensities are available upon request. 
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words, schools reporting high intensity usage of Open Court tended to have a higher average frequency of 
use than was the case for Houghton Mifflin. 

FINDINGS: THE RELATIONSHIP BETWEEN ELA CURRICULUM USE AND API 

The initial analysis found that schools using the Open Court language arts curriculum had higher APIs, on 
average, than schools using alternative curricula. The raw, unadjusted difference between high-intensity 
Open Court versus high-intensity Houghton Mifflin schools was about 17 points on the API scale, which 
represents an “effect size” of just over one-third of a standard deviation. To put this in other terms, 
comparing a randomly chosen high-intensity Open Court school with a randomly chosen high-intensity 
Houghton Mifflin school, there would be about three chances in five that the Open Court school had the 
higher API of the two. Moreover, this relationship persisted after holding constant various demographic 
characteristics (parental education, percent of students eligible for free/reduced lunch, minority 
composition of student body, student mobility, etc.). 

The research team has undertaken some further analyses to examine practices associated with high- 
intensity Open Court usage, which might help to explain why Open Court appears more effective. Since 
not all high-intensity Open Court schools had higher APIs, we sought to determine if other practices 
associated with high-intensity Open Court use might contribute to the higher APIs of some high-intensity 
Open Court schools. 

To investigate this possibility, additional regression models were run that included interactions between 
the high-intensity Open Court curriculum usage indicator and the other educational practice variables 
within the seven broad survey domains. 4 The main results of this exercise suggest that — compared to 
schools that are not intensive users of the Open Court language arts curriculum — high-intensity Open 
Court schools where teachers and principals also report stronger agreement and more frequent use of the 
following practices tend to have higher API scores. 

Teachers across the school agree there is alignment and consistency in curriculum and 
instruction 

• Teachers examine the scope or sequence of curriculum topics. 

• Teachers review a grade-level pacing calendar. 

• There is consistency in curriculum and instruction at the same grade level. 

• There is alignment in curriculum and instruction across different grade levels. 

Both principal and district use assessment data to monitor and evaluate teacher 
practice 

• Principals report using the CAT-6/CST, curriculum program assessments, district developed 
assessments and other commercial assessments data to identify teachers who need instructional 
improvement. 

• Principals report the District uses the CAT-6/CST, curriculum program assessments, district 
developed assessments, and other commercial assessment data to identify teachers who need 
instructional improvement. 

• Principals report using assessment data to determine what professional development teachers 
need to improve in a particular area. 

The association of these practices with higher API scores suggests a school climate with a strong 
academic focus and effective leadership. Teachers across all grades in these schools report frequently 
engaging in multiple practices to ensure the curriculum is coherent between grades and consistent within 
grade levels; principals in these schools report that they and the district use student assessment data from 

4 The results of Open Court and Scott Foresman regressions are listed in columns two and three, respectively, of 
Appendix 3 

Detailed definitions of the items that make up these sub-domain variables (practice areas) can be found in 
Appendix 4. 
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a variety of tests to evaluate teacher practices, identify teachers who need instructional improvement, and 
determine the professional development teachers need in order to improve. 

In addition, these same regressions also identified another statistically significant association: the 
interaction of high-intensity Open-Court use with teachers’ overall agreement that they “have influence 
over hiring decisions” of other teachers and of the school principal. This intriguing finding seems to 
suggest that high intense use of Open Court can go hand-in-hand with a school culture that promotes 
teacher involvement. One possible explanation of this finding is that this perception reflects a strong 
culture of cohesiveness in these schools generally, including around curriculum and assessment practices. 

FINDINGS: MATH CURRICULUM CHOICE AND INTENSITY OF USE BY 
SCHOOLS 

A question with identical wording was asked about the math curriculum program used by the teacher in 
the classroom, with nine state-approved options available and the same range of usage frequency options. 
Again, the average five -point scale response across all of the teachers at a school served as an indicator of 
usage intensity of a given curriculum at each school. The math curriculum program choices (listed in 
alphabetical order) were: Harcourt Brace, Houghton Mifflin, McDougal Little, McGraw Hill, Prentice 
Hall, Progress in Math, Saxon Math, Scott Foresman, Success with MathCoach, and Other. 

The manner in which mathematics curriculum packages were adopted in the schools we studied, and their 
reported usage intensity, stands in strong contrast to the patterns of choice and intensity of usage we 
found for English language arts curriculum packages. In our sample, more different math curriculum 
programs appear to be used within schools and between schools, and smaller percentages of teachers 
report using any one program daily. 

Among the most often used math packages by schools in our study sample were Scott Foresman and 
Harcourt Brace. 

By the above definitions, 18.9% of the schools in our sample were high intensity users of the Scott 
Foresman curriculum, with 46.0% of those schools having all of their teachers reporting daily use of the 
program. 6 Although more of our sample schools used Harcourt Brace’s math program (36.2%) 
intensively, a smaller proportion of those schools (33.3%) had all of their teachers reporting daily use of 
that program. 



Figure 2 - Cross Tabulation of Schools by High Intensity Math Curriculum Usage 
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50 
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50 
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96 
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FINDINGS: THE RELATIONSHIP BETWEEN MATH CURRICULUM AND API 

We could not as clearly establish an association between the use of a specific math program and school 
API. 



On a one-to-five scale, with 1 = Never and 5 = Daily, high-intensity Scott Foresman usage was indicated by an 
average teacher response of 3.8 or higher. High- intensity Harcourt Brace usage was indicated by an average teacher 
response of 3.9 or higher. These thresholds were determined by inspecting the distributions over schools of the 
average teacher responses. Complete distributions of the school-average teacher usage intensities are available upon 
request. 
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CAVEATS 

Results of the preceding analysis suggest that high intensity use of Open Court, coupled with certain 
school and district practices, is associated with higher school API scores for the 257 elementary schools 
in our sample. However, a word of caution is necessary when interpreting these results. The result of this 
analysis implies an association between certain combinations of school practice and API; however, in no 
way does this prove that these practices cause API to increase. 

Schools that are high-intensity Open Court users may differ in various other ways from schools that rely 
more heavily on other language arts packages, possibly including differences on variables not examined 
in the present study. Thus, it is not possible to draw any strong causal inference from the simple 
observation that high-intensity Open Court users have higher APIs. 

In order to develop better evidence for a possible causal influence of Open Court on the API, we used 
statistical models designed to adjust for some other differences between high-intensity Open Court 
schools and those that largely rely on other ELA programs. Although our analysis controlled for many 
demographic factors, such adjustments are never perfect. An ideal study would employ random 
assignment of schools to curricula, but no such investigation could realistically be undertaken on a sample 
as large as the set of schools in our study. In sum, using the nonexperimental data available, it was simply 
not possible to establish causal connections between specific curricula or school practices and API scores. 
We cannot prove or disprove any claim about whether a given school’s API would go up or down or stay 
the same if that school adopted one curriculum program/schooling practice combination over another. 

It is also important to note that estimates of the more general “Open Court effect” (i.e., using the average 
intensity use of Open Court over all teachers as opposed to the high-intensity usage indicator) stemming 
from the original Initial Findings analysis fluctuated somewhat, depending on what other variables were 
included in the model. In order to obtain a more parsimonious model, one that contained only those 
schooling practice variables that accounted for most of the explained variation in API, a stepwise 
procedure was used. While a more parsimonious model that was able to explain virtually the same 
amount of variation in API as the more comprehensive model was obtained, in this model the Open 
Court/API relationship became statistically indistinguishable from 0. 7 Clearly, the interpretation of these 
analyses would be more straightforward if the estimate of the Open Court effect was the same regardless 
of what other variables were included in the model. The fact that we find a statistically significant Open 
Court effect with some combinations of variables and not with others clouds the interpretation. In sum, 
the initial analysis could not find a significant positive relationship between the general use of Open Court 
and API that was robust to various model specifications (i.e., different combinations of schooling practice 
variables), which prompted this extended analysis. 

CONCLUSION 

In the Similar Students, Different Results large-scale survey of elementary schools in California serving 
students in the 25 th to 35 th percentile bandwidth of the state’s School Characteristics Index, the school 
practices that most differentiated the highest- from lowest-performing schools were the following: 

• Prioritizing Student Achievement 

• Implementing a Coherent, Standards-Based Curriculum and Instructional Program 

• Using Assessment Data to Improve Student Achievement and Instruction 

• Ensuring Availability of Basic Instructional Resources 

Our survey yielded responses from 257 principals and over 5,500 K-5 classroom teachers (typically 80% 
or higher from each school). Because we had asked questions about which specific curriculum programs 
the teachers were using for math and for English language arts, we were able to get a solid sense of how 
many different programs were in use by teachers at each school and how often they used each program. 
We were interested in whether any relationship could be detected between school Academic Performance 
Index (API) and schools’ reported choice and intensity of use of any specific curriculum. 

7 The full results of these regressions are included in Appendix 5. 
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For math, a wider variety of curriculum programs appear to be used within schools and between schools, 
and fewer schools report daily use of any particular text. We could not find a clear association between 
use of a specific math program and school API. 

The research team is aware that the choice, use, and success with students of any given curriculum 
program in a district and school is a complex dynamic influenced by many different factors. We are also 
aware that our findings regarding the positive correlation between high-intense Open Court use and 
school API may be provocative. This is why when the association first appeared so strongly in our early 
correlation analysis, we chose to take some extra time to thoughtfully and more fully analyze the finding. 

We are not curriculum specialists and therefore did not study the ways in which the various components 
and attributes of the Open Court EL A curriculum materials differ from the Houghton Mifflin curriculum 
materials, and so we cannot comment on how specifically the curriculum program differences might 
contribute to a more coherent schoolwide curriculum or to a more positive school API. 

However, the results of this extended analysis suggest that for elementary schools with student 
demographics similar to those in our sample, the choice and intense use of Open Court as the schoolwide 
English language arts curriculum program — when combined with implementation of a schoolwide 
coherent, standards-based instructional program and the frequent use of student assessment data to 
improve instruction — appears to be associated with higher school scores on California’s Academic 
Performance Index. 

Within our sample, comparing a randomly chosen high-intensity Open Court school with a randomly 
chosen high-intensity Houghton Mifflin school, there would be about three chances in five that the Open 
Court school had the higher API of the two. Moreover, this relationship persisted after holding constant 
various demographic characteristics (parental education, percent of students eligible for free/reduced 
lunch, minority composition of student body, student mobility, etc.). 

With all the caveats cited above kept in mind, these results provide something to consider in decision 
making about school instructional and curriculum practices and textbook adoption policies. 



This report, Elementary School Curriculum Program and API: A More Detailed Examination, and two supporting 
documents are on the web (address below): 

• Curriculum Program: Technical Appendices 

• California’s Curriculum Adoption Process: Appendix 

These materials are part of a larger study, Similar Students , Different Results: Why Do Some Schools Do 
Better? The study’s Initial Report of Findings and all related materials are on the Web at: 
http://www.edsource.orff/pub abs simstuOS.cfm 



For more information, contact: 

EdSource, Inc. 

520 San Antonio Road, Suite 200 
Mountain View, CA 94040-1217 
edsource@edsource.org 

http://www.edsource.org 

EdSource is a not-for-profit 501(c)(3) organization established in California in 1977. Independent and impartial, EdSource strives 
to advance the common good by developing and widely distributing trustworthy, useful information that clarifies complex K-12 
education issues and promotes thoughtful decisions about California’s public school system. 

Copyright 2006 EdSource, Inc. All rights reserved. Call 650/917-9481 for permission to reprint. All material included in 
this EdSource report of additional findings, released April 2006, is intended solely for the use of our readers. None of this 
material may be used in advertising or for any other commercial purpose. 
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ADDITIONAL FINDINGS RELEASED APRIL 2006 
CURRICULUM PROGRAM: TECHNICAL APPENDICES 



Similar Students , Different Results: 
Why Do Some Schools Do Better? 



APPENDIX I: Curricula Questions from Teacher Survey 

12. Hem frequently do you use the following Language Arts curriculum programs in your 
classroom? 





Once or 

Once or Twice A Few 

Twice a Times 

Daily a Week Mouth a Year Never 


a. Harcoiut Brace Spelling 




b. Houghton Mifflin 




c. McGraw Hill MacMillan's Readmg 




d. Open Court 




e. Phonics SRA 




f. Scholastic Phonics 




g. Scott Foresinan 




h. Write Source Language Program 




i. Other (please specify): 





13. How frequently do you use the following Math curriculum programs in your classroom? 





Once or 

Once or Twice A Few 

Twice a Times 

Daily a Week Mouth a Year Never 


a. Harcourt Brace 




b. Houghton Mifflin 




c. McDougal Little 




d. McGraw Hill 




e. Prentice Hall 




f. Progress in Math 




g. Saxon Math 




h. Scott Foresman 




i Success with MathCoach 




1 . Other / please specify) : 
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APPENDIX 2A: Descriptive Statistics of Reported Language Arts Curriculum Usage Frequency by 
High Intensity Usage Category 



Language Arts Program 


Number 

of 

Schools 


Average 
Response 
(Scale of 1 to 5) 


Standard 

Deviation 


Minimum 


Maximum 


High Intensity Open Court Usage = 1 & High Intensity Houghton Mifflin Usage = 0 


Harcourt Brace Spelling 


72 


1.13 


0.27 


1.00 


2.33 | 


Houghton Mifflin 


72 


1.30 


0.43 


1.00 


3.00 | 


McGraw Hill/MacMillan's Reading 


72 


1.21 


0.38 


1.00 


3.25 


Open Court 


72 


4.96 


0.09 


4.59 


5.00 


Phonics SRA 


72 


1.82 


0.54 


1.00 


3.00 | 


Scholastic Phonics 


72 


1.21 


0.33 


1.00 


2.80 


Scott Foresman 


72 


1.26 


0.41 


1.00 


2.60 


Write Source Language Program 


72 


1.26 


0.45 


1.00 


3.54 | 


Other 


72 


2.75 


1.18 


1.00 


5.00 | 


High Intensity Open Court Usage = 0 & High Intensity Houghton Mifflin Usage = 1 


Harcourt Brace Spelling 


177 


1.25 


0.37 


1.00 


3.38 | 


Houghton Mifflin 


177 


4.86 


0.23 


4.00 


5.00 


McGraw Hill/MacMillan's Reading 


177 


1.24 


0.36 


1.00 


3.14 


Open Court 


177 


1.11 


0.27 


1.00 


2.88 | 


Phonics SRA 


177 


1.34 


0.47 


1.00 


4.31 | 


Scholastic Phonics 


177 


1.33 


0.40 


1.00 


2.83 


Scott Foresman 


177 


1.22 


0.39 


1.00 


3.00 | 


Write Source Language Program 


177 


1.32 


0.46 


1.00 


3.13 | 


Other 


173 


3.26 


1.16 


1.00 


5.00 ! 


High Intensity Open Court Usage = 0 & High Intensity Houghton Mifflin Usage = 0 


Harcourt Brace Spelling 


16 


1.18 


0.21 


1.00 


1.71 


Houghton Mifflin 


16 


3.01 


0.66 


1.72 


3.87 


McGraw Hill/MacMillan's Reading 


16 


1.66 


0.78 


1.00 


3.67 


Open Court 


16 


2.36 


1.22 


1.00 


4.11 1 


Phonics SRA 


16 


1.56 


0.85 


1.00 


4.22 


Scholastic Phonics 


16 


1.45 


0.61 


1.00 


3.00 | 


Scott Foresman 


16 


1.22 


0.30 


1.00 


2.00 | 


Write Source Language Program 


16 


1.26 


0.48 


1.00 


3.00 j 


Other 


16 


3.67 


1.06 


1.00 


5.00 
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APPENDIX 2B: Descriptive Statistics of Reported Math Curriculum Usage Frequency by High 
Intensity Usage Category 



Math Program 


Number 

of 

Schools 


Average 
Response 
(Scale of 1 to 5) 


Standard 

Deviation 


Minimum 


Maximum 


High Intensity Scott Foresman Usage = 1 & High Intensity Harcourt Brace Usage = 0 


Harcourt Brace 


50 


1.08 


0.17 


1.00 


1.88 


Houghton Mifflin 


50 


1.24 


0.40 


1.00 


2.69 ! 


McDougal Little 


50 


1.01 


0.04 


1.00 


1.23 ! 


McGraw Hill 


50 


1.08 


0.14 


1.00 


1.47 | 


Prentice Hall 


50 


1.03 


0.06 


1.00 


1.25 | 


Progress in Math 


50 


1.02 


0.11 


1.00 


1.75 


Saxon Math 


50 


1.16 


0.34 


1.00 


2.30 


Scott Foresman 


50 


4.85 


0.26 


3.87 


5.00 


Success with MathCoach 


50 


1.12 


0.23 


1.00 


2.00 


Other 


49 


2.66 


0.95 


1.00 


5.00 ! 


High Intensity Scott Foresman Usage = 0 & High Intensity Harcourt Brace Usage = 1 


Harcourt Brace 


96 


4.77 


0.25 


4.05 


5.00 


Houghton Mifflin 


96 


1.31 


0.38 


1.00 


2.82 


McDougal Little 


96 


1.01 


0.04 


1.00 


1.21 


McGraw Hill 


96 


1.12 


0.20 


1.00 


2.25 ! 


Prentice Hall 


96 


1.03 


0.11 


1.00 


1.75 ; 


Progress in Math 


96 


1.01 


0.06 


1.00 


1.47 


Saxon Math 


96 


1.06 


0.27 


1.00 


3.40 ; 


Scott Foresman 


96 


1.09 


0.15 


1.00 


1.75 


Success with MathCoach 


96 


1.04 


0.14 


1.00 


2.00 


Other 


94 


2.57 


1.14 


1.00 


4.67 ! 


High Intensity Scott Foresman Usage = 0 & High Intensity Harcourt Brace Usage = 0 


Harcourt Brace 


119 


1.20 


0.51 


1.00 


3.94 


Houghton Mifflin 


119 


2.87 


1.83 


1.00 


5.00 ; 


McDougal Little 


119 


1.03 


0.14 


1.00 


2.33 i 


McGraw Hill 


119 


1.47 


1.12 


1.00 


5.00 ! 


Prentice Hall 


119 


1.03 


0.12 


1.00 


2.00 


Progress in Math 


119 


1.08 


0.36 


1.00 


3.75 ! 


Saxon Math 


119 


2.32 


1.82 


1.00 


5.oo ; 


Scott Foresman 


119 


1.17 


0.42 


1.00 


3.80 ; 


Success with MathCoach 


119 


1.03 


0.09 


1.00 


1.60 i 


Other 


117 


2.84 


1.28 


1.00 


5.00 | 
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APPENDIX 3: Regressions of 2005 Growth API on Domain I Composite Variables with Curriculum 
Interactions 





Composite 
Variables with 
High Intensity 
Open Court/ 
Scott Foresman 
Indicators 


Composite 
Variables with 
High Intensity 
Open Court 
Indicator 
Interactions 


Composite 
Variables with 
High Intensity 
Scott Foresman 
Indicator 
Interactions 


Teachers report alignment and consistency in curriculum and 
instruction 


6.77 

(1.82)* 


1.97 

(0.54) 


4.88 

(1.29) 


Principal and district have clear and shared expectations that 
guide coherence around the curriculum 


7.20 

(2.53)** 


7.69 

(2.55)** 


5.34 

(1.71)* 


District addresses instructional needs of EL students 


-0.37 

(0.05) 


O OS 
UJ 
ISi UJ 


1.53 

(0.18) 


District, principals, and teachers utilize state standards to 
guide curriculum and instruction 


4.13 

(1.23) 


5.95 

(1.64) 


5.95 

(1.57) 


Math and language arts are integrated with other subjects. 


6.43 

(2.05)** 


6.32 

(1.73)* 


5.95 

(1.74)* 


Open Court high-intensity usage (OCH) 


9.11 

(1.23) 


62.40 

(1.61) 




Scott Foresman high-intensity usage (SFH) 


13.62 

(1.97)** 




57.62 

(1.43) 


Teachers report alignment and consistency in curriculum and 
instruction/OCH interaction 




17.61 

(2.16)** 




Use of Open Court language arts curriculum program/ 
OCH interaction 








The school and district places a priority on consistency of 
curriculum/OCH interaction 




-0.77 

(0.11) 




Use of Scott Foresman math curriculum program/ 
OCH interaction 








District addresses instructional needs of English learner 
students/OCH interaction 




-20.73 

(1.46) 




District, principals, and teachers utilize state standards to 
guide curriculum and instruction/OCH interaction 




-5.02 

(0.66) 




Math and language arts are integrated with other 
subjects/OCH interaction 




-1.72 

(0.22) 




Teachers report alignment and consistency in curriculum and 
instruction/SFH interaction 






13.74 

(1.52) 


Principal and district have clear and shared expectations that 
guide coherence around the curriculum/SFH interaction 






11.75 

(1.73)* 


District addresses instructional needs of EL students/SFH 
interaction 






-16.47 

(1.08) 


District, principals, and teachers utilize state standards to 
guide curriculum and instruction/SFH interaction 






-4.02 

(0.64) 


Math and language arts are integrated with other 
subjects/SFH interaction 






-3.62 

(0.46) 


* significant at 10%; ** significant at 5%; *** significant at 1%. Robust t statistics in parentheses. Open Court and Scott Foresman 
high-intensity usage indicators are defined as follows: 0=No and l=Yes. 
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Appendix 3 (continued) - Regressions of 2005 Grovrth API on Domain 1 Composite Variables with Curriculum Interactions 




Composite 
Variables with 
High Intensity 
Open Court/ 
Scott Foresman 
Indicators 


Composite 
Variables with 
High Intensity 
Open Court 
Indicator 
Interactions 


Composite 
Variables with 
High Intensity 
Scott Foresman 
Indicator 
Interactions 


Parent education level: percent high school graduate or 
some college 


-32.39 

(1.17) 


-28.34 

(1.03) 


-39.41 

(1.44) 


Parent education level: percent college graduate or graduate 
school 


54.87 

(1.04) 


40.83 

(0.75) 


48.21 

(0.90) 


School enrollment 


-0.00 

(0.12) 


0.00 

(0.03) 


0.00 

(0.07) 


Percent African American 


-0.09 

(0.29) 


-0.11 

(0.36) 


0.04 

(0.12) 


Percent Asian 


0.52 


0.46 


0.51 


(1.19) 


(0.99) 


(1.12) 


Percent Hispanic 


-0.13 

(0.50) 


-0.16 

(0.63) 


-0.14 

(0.54) 


Percent eligible for free/reduced meals 


-0.46 

(1.53) 


-0.43 

0.48) 


-0.51 

(1.73)* 


Percent English learners 


-0.51 

(2.16)** 


-0.62 

(2.75)*** 


-0.43 

(1.78)* 


Student mobility 


2.69 

(3.55)*** 


2.90 

(3.81)*** 


2.46 

(3.32)*** 


Percent enrolled in migrant education programs 


-0.34 

(1.33) 


-0.36 

(1.45) 


-0.26 

(1.00) 


Constant 


527.27 

(7.23)*** 


494.29 

(6.69)*** 


547.98 

(7.72)*** 


Observations 


248 


248 


248 


Adjusted R-squared 


0.2428 


0.2424 


0.2510 


* significant at 10%; ** significant at 5%; *** significant at 1%. Robust t statistics in parentheses. Reference groups are defined as 
follows: for parent education level - less than high school; for minority percent of enrollment - percent of students that are white or 
American Indian, Filipino, Pacific Islanders (note: latter three categories make up less than 4% of students in school sample). 
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Appendix 4: Significant Composite Variables in Curriculum Open Court/Practice Interaction Analysis 



Composite Variable — Teachers report alignment and consistency in curriculum and instruction. 
Domain - Implementing a Coherent, Standards-Based Curriculum and Instructional Program 


Teacher 
Survey Item 


Item Description 


4c 


Teachers examine the scope or sequence of curriculum topics 


4d 


Teachers review a grade-level pacing calendar 


5c 


There is consistency in curriculum and instruction at the same grade level 


! 5d 


There is alignment in curriculum and instruction across different grade levels 



Composite Variable - Teachers have influence over hiring decisions. 

Domain - District and school encourage teacher collaboration and build educator capacity 


Teacher 
Survey Item 


Item Description 


7e 


Influence of teachers at school in the hiring of new teachers 


7f 


Influence of teachers at school in the hiring of new principals 



Composite Variable -District and principals use assessment data to monitor and evaluate teacher 
performance. 

Domain — Using Assessment Data to Improve Student Achievement and Instruction 


Principal 
Survey Item 


Item Description 


25ag 


I use CAT-6/CST data to identify teachers who need instructional improvement 


25bc 


District uses CAT-6/CST data to evaluate teachers’ practices 


27cg 


I use curriculum program assessment data to identify teachers who need instructional 
improvement 


28c 


District uses curriculum program assessment data to evaluate teachers’ practices 


29cg 


I use district developed assessment data to identify teachers who need instructional 
improvement 


29dc 


District uses district developed assessment data to evaluate teachers’ practices 


30cg 


I use other commercial assessment data to identify teachers who need instructional 
improvement 


30dc 


District uses other commercial assessment data to evaluate teachers’ practices 


32e 


I use assessment data to determine what professional development teachers need to 
improve in a particular area 
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Appendix 5: Regressions of 2005 Growth API on Domain I Composite Variables 







Set of Composite 




All Domain 1 Composite 


Variables That Account 




Variables 


for Majority of 
Explained Variance i 


Math and language arts are protected from unnecessary 


-2.59 




interruptions 


(3.47) 




Math and language arts are integrated with other subjects 


4.60 

(4.19) 


6.31 1 

(2.03)** 


Proportion of teachers using dominant language arts 


-1.83 




curriculum program 


(2.54) 




Proportion of teachers using dominant math curriculum 


2.61 




program 


(2.20) 




Teachers report alignment and consistency in curriculum 


6.84 


6.59 


and instruction 


(3.44)** 


(1.77)* 


Teachers are consistent in their use of curriculum 


-3.74 




planning and materials 


(4.00) 




District, principals, and teachers utilize state standards to 


5.16 


4.39 1 


guide curriculum and instruction 


(3.73) 


(1.30) j 


Teachers meet or exceed the amount of time 


2.76 




recommended by the state frameworks for language arts 


(6.95) 




Teachers meet or exceed the amount of time 


2.10 




recommended by the state frameworks for math 


(6.95) 




School-level intensity of Open Court use among teachers 


4.77 

(2.08)** 


3.07 

(1.61) 


Use of Phonics SRA language arts curriculum program 


0.62 

(5.47) 




Use of Scholastic Phonics language arts curriculum 


0.02 




program 


(7.50) 




Use of Houghton Mifflin math curriculum program 


1.85 

(1.99) 




Use of Saxon math curriculum program 


-2.66 

(2.83) 




School-Level Intensity of Scott Foresman use among 


2.60 


3.41 


teachers 


(1.91) 


(1.90)* 1 


School uses a standards-based report card 


2.03 

(3.06) 




District addresses instructional needs of English learner 


5.19 


-0.03 | 


students 


(7.48) 


(0.00) ! 


Principal and district have clear and shared expectations 


6.73 


7.10 


that guide coherence around the curriculum 


(3.17)** 


(2.51)** 


District, principals, and teachers utilize state standards to 


-0.57 




guide curriculum and instruction 


(3.24) 




Principal acts as a knowledgeable source concerning 


5.67 




standards and curriculum 


(5.17) 




In the last four years, school has implemented a new 


-2.59 




program for English learner students 


(5.94) 




* significant at 10%; ** significant at 5%; *** significant at 1%. Robust t statistics in parentheses. Reference groups are ! 

defined as follows: for parent education level - less than high school; for minority percents - percent of students that are 
white or American Indian, Filipino, Pacific Islanders (the latter three categories make up less than 4% of students in school 


sample). 
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Appendix 5 (continued) - Regressions of 2005 Growth API on Domain 1 Composite Variables 






Set of Composite 




All Domain 1 Composite 


Variables That Account 




Variables 


for Majority of 
Explained Variance 


Parent education level: percent high school graduate or 


-33.63 


-30.39 


some college 


(28.05) 


(1.10) 


Parent education level: percent college graduate or 


38.35 


57.31 


graduate school 


(53.92) 


(1.08) 


School enrollment 


-0.01 

(0.02) 


-0.00 

(0.13) 


Percent African American 


-0.29 

(0.30) 


-0.12 

(0.38) 


Percent Asian 


0.65 

(0.44) 


0.48 

(1.11) 


Percent Hispanic 


-0.11 

(0.26) 


-0.15 

(0.58) 


Percent eligible for free/reduced meals 


-0.51 

(0.31)* 


-0.45 

(1.51) 


Percent English learners 


-0.66 

(0.26)** 


-0.51 

(2.15)** 


Student mobility 


2.87 

(0.80)*** 


2.63 

(3.44)*** 


Percent enrolled in migrant education programs 


-0.39 

(0.25) 


-0.33 

(1.30) 


Constant 


459.00 

(97.66)*** 


524.13 
(1 17)*** 


Observations 


243 


248 


Adjusted R-squared 


0.2530 


0.2485 


* significant at 10%; ** significant at 5%; *** significant at 1%. Robust t statistics in parentheses. Reference groups are 
defined as follows: for parent education level - less than high school; for minority percent of enrollment - percent of students 
that are white or American Indian, Filipino, Pacific Islanders (note: latter three categories make up less than 4% of students 


in school sample). 
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